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A SBIR (Small Business Innovative Research) contract was 
awarded to Analytics to develop a system which uses the faint 
magnetic fields generated by the mental processes of the brain as 
another route for computer control. The emphasis of this project is 
not on the direct control of a computer workstation by reading the 
operator’s EEG signals, but to control the workstation, or better, to 
anticipate the user’s needs with the combination of eye data, EEG 
data and task or workload data. 

Analytics, through a previous SBIR, developed a workstation 
which used the operator's eye movements and position to determine 
the placement of the cursor on a computer screen. This paper will 
first provide a brain wave sensing technology overview and an 
introduction into the known rhythms or signals generated by the 
brain. This will be followed with a descriptive explanation of OASIS 
(Ocular Attention Sensing Interface System) and its intended 
integration into the proposed testbed. 

Introduction 

With the ever growing computer processing speed comes the 
growing informational exchange between man and machine. One 
interface design aspect deals primarily with the presentation or 
selection of information to the user. Another interface design aspect 
concentrates on control mechanisms such as the mouse to 
manipulate the data available to the user. This project's basic 
foundation is that both aspects should be addressed simultaneously. 

Most advanced man machine interfaces are directed toward the 
cockpit. Complex machines have been developed to simulate the 
actual sensations and perceptions experienced by a pilot during 
flight. Should advanced interfaces be confined to just the super 
cockpit? Could such interfaces benefit the scientist or engineer in 
recreating known concepts? Clearly, many scientific problems, 
especially those that can be represented in three dimensions, would 
benefit by a greater interaction with the computer. 
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Electrical/Magnetic Sensing Techniques Overview 


Electroencephalography (EEG) is probably the most familiar 
recording technique. EEG's are routinely used by the the medical 
community in diagnosing certain sleep or mental disorders. Also, the 
EEG along with heart rate and perspiration detection instruments are 
used by interrogative polygraphy or colloquially known as "lie 
detection". Alpha rhythms were first discovered by Berger (1929). It 
is interesting to note that alpha rhythms were discovered with 
galvanometers (a magnetic sensing device). It did not become 
feasible to record alpha rhythms until the introduction of the 
vacuum tube amplifier in the 1940's. A major problem with the early 
vacuum tubes were their inherent noise. Today's solid state low- 
noise amplifiers are capable of amplifying signals from DC up into the 
kilohertz regions with little degradation in the signal. In fact, the 
signals amplified by these devices are so clean that the principle 
noise source now comes from the neighboring mental activity of the 
area under investigation. The signals being detected by the EEG scalp 
probes are not only from the activity directly under the probe but 
also from other distant sources. EEG readings then are the average 
effect of several activity sources. 

The signals seen by the EEG are frequencies which range from 
between 1 cycle per second to 13 cycles per second. Different 

frequency bands represent different states a person is undergoing. 
An EEG can easily determine whether a person is at rest, 
daydreaming, sleeping, or mentally active. 

Probably the most familiar type of EEG signal is the alpha 
rhvthm . Its range is generally between 8 to 12 cycles per second. An 
alpha rhythm is characteristic of a person in a relaxed state with the 
eyes closed but not necessarily sleeping. Its signal is greatest in 
amplitude over the posterior portions of the head, and it occurs in 

spindles with varying durations. The spindles can be inhibited by 

having a person open his eyes. This should indicate that the alpha 

rhythm is directly correlated with visual activity. However, after a 
person has had his eyes open and slips into a boring situation, it is 
possible to have the alpha rhythms reappear even with the subject's 
eyes open. 

Another interesting EEG reading is the beta rhvthm . It is 
characteristic of signals of 13 cycles per second and above. Beta 
activity is present when an individual has her eyes open and is 
mentally active. The signal is present in the anterior quadrants of the 
head. 
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Theta rhythms are characteristic of signals in the frequency 
band between 4 and 7 cycles per second. These rhythms are found in 
both adults and children under emotional stress. The delta rhythms 
are present during sleep and are in the 1 to 3 cycle per second 
frequency range. Delta rhythms appear during sleep and characterize 
the various sleep levels. Due to their dependence on the sleep state, 
they are not useful for this project. 

Other signals which can be detected by EEG’s are those that are 
evoked by certain stimuli. Evoked potentials, then, are time-locked to 
specific events. Because of their direct relation to cognitive factors 
the Nl, P2, and P3 are particularly interesting to this project. The 
identifying characteristic for the evoked potential is easily 
understood by noting that the preceding letter signifies whether the 
signal is negative (N) or positive (P). The number following the letter 
indicates how long after the stimulus is given that a response is 
expected. Therefore, for Nl the signal is negative and occurs 100 
milliseconds after the stimulus. 

First discovered by Sutton (1963) is the classic P300 ( P3 ) 
phenomenon. The fact that a component of the auditory related 
potential occurred about 300 msec after an unexpected stimulus was 
quite exciting This led to more discoveries of other event related 
responses. Due to the low frequency characteristic of P3, it is useful 
only in slowly developing situations. P3 is evoked by surprising or 
unexpected occurrence of both visual and auditory stimuli. However, 
for odd visual stimuli, the response is usually more on the order of 
400 msec. Its long reaction time can be understood when we imagine 
a confrontation with a surprising situation ... we absorb the stimulus 
... we analyze the stimulus ... we detect an anomaly ... we react. 

The N 1 evoked potentials are activated by auditory stimuli. 
Magnetic studies by Pellizone (1984) showed that the source of Nl is 
in the auditory cortex. It has been demonstrated that the amplitude 
of these signals are strongly related to the subject’s level of attention. 
Unfortunately, the characteristics of this signal vary across 
individuals. 

Almost anything stated of Nl is also true for P2 . except that far 
less is known of P2 than Nl. Magnetic studies of this phenomenon 
have shown physically separated activity sources within the brain. 

With every electrical field there is an associated magnetic field. 
Unlike EEG which requires that probes be placed on the scalp, 
magnetoencephalogram (MEG) sensors are placed reasonably close to 
the scalp but not required to touch. The principle advantage this 
offers is the elimination of unwanted "skin" noise which poses a large 
problem for EEG recordings. As was indicated earlier, both theoretical 
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and experimental EEG studies have shown that different tissues and 
bone surrounding the brain attenuate or 'smear' the potentials that 
reach the scalp (Geisler and Gerstein, 1961; De Lucchi et al., 1962; 
Cooper et al., 1965). However, MEG studies have demonstrated that 
concentric layers in the head do not affect the magnetic fields 
produced by sources (dipoles). Therefore, MEG is not subject to 
'smearing' and can thereby detect sources which occupy small 
regions (Grynszpan and Geselowitz, 1973). 

Not until the introduction of a Superconducting Quantum 
Interference Device (SQUID) could magnetic fields emanating from 
the brain be detected. The brain’s magnetic field has a strength on 
the order of magnitude of 10'8 Gauss (G) (Reite et al., 1976). In 
comparison, the earth's magnetic field has a strength in the order of 
0.5 G (Geselowitz, 1979). Strong fields such as the earth’s can be 
filtered with a gradiometer. For a SQUID to detect magnetic fields in 
the order of magnitude of 10' 10 G, the sensors must operate in the 
superconducting region. Not until recent breakthroughs in 
superconducting materials operating at near room temperatures 
(Chen, 1987) was this possible without the use of liquid helium to 
achieve the superconducting effect. Supercooling demanded 
cryogenics which made any research with SQUID's costly prohibitive. 
At the writing of this paper only one commercial company, that the 
author is aware of, has been able to fabricate metals for commercial 
use. But, as with other electronic technologies, fabricating 
superconducting materials for commercial use is expected to soon 
become commonplace. 

The Eye/Brain/Task Testbed 

Computer interfaces, as presently known, are all basically 
passive. That is, a computer does not respond to the user without the 
user's request. Ideally, an intelligent system would know when 
intervention is appropriate. The passive role of "ready servant" 
requires that an operator’s needs are anticipated. As an analogy to 
the ready servant, consider the skilled nurse assisting the surgeon 
anticipating the needs of the surgeon before the request is made. As 
the computer begins to take on a more active role, the need for the 
machine to know the operator’s activity and intentions becomes 
essential. This phenomenon we experience everyday, but, may never 
fully realized. In carrying conversations with other individuals we 
unconsciously notice the person's facial expressions, posture, eye 
contact, and variations in speech to anticipate what the person’s 
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intentions are and what he will be saying. Being able to detect these 
subtleties using optics and computers would be extremely difficult. 
But, it is believed that using instruments such as MEG or EEG which 
can estimate a person’s mental state, practical steps are being taken 
toward knowing a person's intention with a computer. The 
Eye/Brain/Task (EBT) project will be a testbed to test these theories. 

Before an explanation for how the various electro/megneto 
recording techniques will be used on this project, a description of the 
Ocular Attention Sensing Interface System (OASIS) should be 
provided. OASIS is comprised of a testbed and prototype eye/voice 
control system. Basically, a low intensity infrared light is shined onto 
a person’s eyes. The pupils of the eye reflect the light to an 
oculometer which determines eye position. This information is then 
fedback to the monitor in the form of a cursor. Therefore, this 
prototype allows an operator to move a cursor on the screen by 
simply moving the eyes. A voice recognizer also allows the operator 
to give verbal commands. By integrating eye data with brain wave 
activity, it then becomes possible to determine where a person is 
looking in relation to the screen and the person's associated level of 
attention within the region. 

To fully complete the required components to determine a 
person’s intention, an indicator of the workload or task is still 
required. The full integration of the three components, eye, brain, 
and task can best be explained with an example. Imagine a scene as 
depicted in figure 1. The object of this task is to direct as many 
vehicles as possible through the obstacles to reach the other side. 
"Looking" at a vehicle and giving it the verbal command "GO" propels 
it forward toward the obstacles. Another vehicle can then be selected 
and also given the command to proceed. By this time it probably 
becomes necessary to service the first vehicle and direct it to make 
either a "LEFT" or "RIGHT" turn. Another vehicle is then selected and 
also given the command to proceed forward. What has been 
developed here is a rich environment which allows a researcher to 
correlate task information with brain activity and visual information. 
Audio stimuli can also be interjected. 
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Figure 1. Vehicle manuevering task 
Conclusion 

The Phase II effort will concern itself with the development of 
a prototype EBT testbed, and, through applied research and 
development, the refinement and optimization of the system. The 
principle objective of the proposed Phase II effort is to develop a 
laboratory testbed that will provide a unique capability to elicit, 
control, record, and analyze the relationship of operator task loading, 
operator eye movement, and operator brain wave data in a computer 
system environment. Additionally, the testbed will have the 
capability to serve as the vehicle for demonstrating computer control 
using brain waves at a future time. 

Acronyms 


DC 

direct current 


EBT 

Eye/Brain/Task 


EEC 

electroencephalography 


G 

Gauss 


MEG 

magnetoencephalography 


OASIS 

Ocular Attention Sensing 

Interface System 

SBIR 

Small Business Innovative 

Research 

SQUID 

Superconducting Quantum 

Interference Device 
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